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ABSTRACT:  The  Live  Fly  Phase  (LFP)  of  the  Systems  Integration  Test  (SIT)  was  executed  by  the  Joint  Advanced 
Distributed  Simulation  (JADS)  Joint  Test  Force  (JTF)  and  the  46th  Test  Wing  at  Eglin  AFB,  FL  during  1997.  The 
purpose  of  the  SIT  is  to  evaluate  the  utility  of  using  advanced  distributed  simulations  (ADS)  to  support  cost-effective 
testing  of  an  integrated  missile  weapon/launch  aircraft  system  in  an  operationally  realistic  scenario.  The  SIT  missions 
simulate  a  single  shooter  aircraft  launching  an  air-to-air  missile  against  a  single  target  aircraft,  (reference  [1]) 

In  the  LFP,  the  shooter  and  target  were  represented  by  live  aircraft  and  the  missile  by  a  simulator.  ADS  techniques 
were  used  to  link  two  live  F-16  fighter  aircraft  flying  over  the  Eglin  Gulf  Test  Range  to  the  AMRAAM  AIM-120 
hardware-in-the-loop  (HWIL)  simulation  facility  at  Eglin.  In  order  to  successfully  integrate  these  assets  for  a  near 
real-time  test,  the  JADS  team  learned  several  lessons  during  the  risk  reduction  and  test  execution  phases.  The  lessons 
highlighted  here  concern  test  control  aspects,  computer  processing,  and  telemetry  issues.  Control  of  a  distributed  test 
dealt  with  tactical  aircraft  control,  scenario  and  data  collection  decisions,  collocation  of  critical  project  personnel,  and 
voice  communications.  Computer  processing  lessons  dealt  with  simulated  GPS  data,  pre-processing  live  GPS  data 
from  several  aircraft  pods,  creation  of  an  aircraft  to  HWIL-missile  interface,  and  contingency  planning  for  real-time 
malfunctions.  Telemetry  issues  concerned  aircraft  and  terrain  shielding,  and  an  implementation  to  handle  random 
sensor  dropouts. 

These  lessons  would  be  applicable  for  other  projects  when  coupling  live  and  virtual  assets  for  evaluation  of  fire  control 
radars  or  precision  guided  munitions.  Many  lessons  on  control  and  processing  also  apply  to  simulation  tests  which 
link  distributed  facilities. 


1.  Background  for  Using  Advanced  Distributed  Simulation 


Testing  of  aircraft  and  missile  systems  has  evolved  into  a  combination  of  mutually  supportive  test  events, 
such  as  open  air  flight  tests,  installed  system  tests,  hardware-in-the-loop  (HWIL)  evaluations,  and  digital 
simulations.  Each  event  would  provide  some  insight  for  different  objectives  or  measures  of  effectiveness. 
Their  data  and  results  would  support  predictions  of  performance  for  the  next  phase  of  testing,  but  for  the 
most  part,  these  events  have  been  independent  of  each  other. 


Figure  1.1  T&E  Measurement  Facilities 

(see  reference  [2]) 

In  the  last  decade,  computer  processing  speed  has  increased  drastically,  and  technology  for  connecting  Local 
Area  Networks  (LANs)  into  Wide  Area  Networks  (WANs)  has  become  readily  available.  In  an  effort  to 
utilize  these  advances,  the  testing  community  is  investigating  a  new  tool  to  enhance  test  and  evaluation 
(T&E)  realism  and  productivity,  while  possibly  tying  together  some  test  events  in  a  distributed  network. 

The  JADS  Joint  Test  Force  (JTF)  was  chartered  to  investigate  the  utility  of  ADS  technologies  in  support  of 
Developmental  Test  and  Evaluation  (DT&E)  and  Operational  Test  and  Evaluation  (OT&E).  (see  reference  [3]) 
The  JTF  has  been  investigating  three  representative  slices  of  the  T&E  spectrum.  The  System  Integration  Test 
(SIT)  has  conducted  two  air-to-air  missile  phases,  while  the  End-to-End  (ETE)  team  will  explore  ADS  utility 
for  Command,  Control,  Communication,  Computers  and  Intelligence  (C4I),  and  the  Electronic  Warfare  (EW) 
team  will  investigate  distributed  EW  testing,  (reference  [1]) 

1.1  Overview  of  the  SIT  Live  Fly  Phase 

As  stated  in  the  abstract,  the  purpose  of  the  SIT  is  to  evaluate  the  utility  of  using  ADS  to  support  cost- 
effective  testing  of  an  integrated  missile  weapon/  launch  aircraft  system  in  an  operationally  realistic 
scenario.  The  Live  Fly  Phase  (LFP)  combined  an  open-air  flight  test  aspect  (using  two  F-16  aircraft  over  the 
Eglin  Gulf  Test  Range)  with  an  HWIL  facility  (an  AIM-120  Missile  Simulation  Laboratory  (MISILAB)  also  at 
Eglin).  These  systems  were  chosen  because  of  their  high-technology,  well  instrumented,  and  readily 
available  test  platforms,  which  when  linked  together,  are  representative  of  a  wide  class  of  fighter  aircraft 
with  precision  guided  munitions. 

ADS  networking  could  benefit  other  projects  which  evaluate  the  effectiveness  of  fire  control  radars  interfaced 
with  air-to-air,  surface-to-air,  and  air-to-ground  missiles. 

1.2  Aircraft  Data 

In  the  LFP,  a  single  F-16  shooter  acquired  and  tracked  a  single  F-16  target  The  shooter  aircraft  carried  an 
instrumented  pod  which  simulated  an  AIM-120  missile.  Each  aircraft  also  carried  two  instrumented  Global 
Positioning  System  (GPS)  pods  (which  were  the  four-channel  prototype  RAJPO  HDIS  models  with  real-time 
transmitters).  Both  the  aircraft  and  the  pods  transmitted  data  to  the  ground  in  real-time,  where  telemetry 


(TM)  antennas  on  the  Eglin  coastline  would  receive  and  pass  the  data  through  to  the  Central  Control  Facility 
(CCF).  This  data  flow  is  shown  on  the  upper  portion  of  Figure  1.2  (on  next  page). 

1.3  CCF  Processing 

The  CCF  received  and  processed  several  sources  of  data. 

Each  aircraft  sent  data  from  its  Inertial  Navigation  System  (INS)  at  a  50  Hertz  rate  (i.e.,  50  updates  per 
second),  as  well  as  its  altimeter  and  weapons  select  TM  data.  The  two  GPS  pods  on  each  aircraft  telemetered 
their  raw  receiver  measurements  in  a  1-8-1  mode  (i.e.,  1  navigation  solution  message,  8  pseudorange 
measurements,  and  1  status  message).  Therefore,  the  4  GPS  pods  transmitted  10  Hz  data  apiece. 
Meanwhile,  the  CCF  also  processed  GPS  ground  reference  receiver  measurements  (from  atop  their  building) 
in  order  to  differentially  correct  the  data  from  the  dynamically  moving  aircraft  pods.  The  standard  operating 
procedure  for  the  Eglin  range  also  mandated  the  processing  of  FPS-16  ground  tracking  radars,  which  sent 
azimuth,  elevation,  and  range  measurements  on  each  aircraft  to  the  CCF  at  a  10  Hz  rate.  The  CCF  pre- 
processed  these  sensor  data  through  different  real-time  computers,  mostly  1980's  vintage  Digital  Equipment 
Corp  VAX  mainframes. 

These  computers  sent  the  Time  Space  Position  Information  (TSPI)  through  to  the  TSPI  Data  Processor  (TDP). 
This  TDP  combined  the  sources  of  INS,  GPS,  and  FPS-16  data  into  a  time-correlated  entity  state  solution  for 
each  aircraft. 

Meanwhile,  on  a  separate  TM  data  frequency  from  the  TSPI,  the  CCF  also  processed  the  airborne  missile  pod 
data.  The  missile  TM  stream  was  processed,  and  the  data  were  time-stamped  (based  on  tire  missile  clock 
counter)  to  an  IRIG-B  time  standard. 

1.4  ADS  Conversion  and  Interface 

Now  in  order  for  all  this  instrumented,  open-air  range  data  to  work  with  any  simulation  language,  it  had  to 
be  converted  into  a  standardized  message  format.  For  this  ADS  test,  JADS  chose  to  use  Distributed 
Interactive  Simulation  (DIS)  Protocol  Data  Units  (PDUs).  Both  the  missile  data  and  the  TDP  solution  data 
then  were  processed  through  a  "network  interface  unit";  this  took  the  range  data  and  converted  it  into  DIS 
PDUs.  This  conversion  process  took  place  in  a  Digital  Equipment  Corp.  Alpha  600  machine  in  the  CCF,  and 
was  deconverted  in  another  Alpha  machine  in  the  MISILAB  (shown  in  the  center  of  Figure  1.2). 

1.5  MISILAB  Simulation  Processing 

After  the  MISILAB  Alpha  received  the  DIS  PDUs,  that  network  interface  unit  converted  the  message  into  the 
format  used  by  the  AMRAAM  HWIL  simulation.  It  also  resynchronized  the  shooter  and  target  data,  and 
interpolated  the  data  up  to  the  600  Hz  frame  rate  used  by  the  simulation. 

The  MISILAB  then  pre-processed  and  recorded  the  data  out  of  the  Alpha  and  into  different  mainframes.  The 
MISILAB  split  off  the  live  feed  from  the  target  aircraft  to  an  anechoic  chamber,  where  22  antenna  horns 
would  emulate  the  target  position  and  radio  frequency  (RF)  return  signal  strength.  The  live  feed  for  the 
shooter  data  was  fed  to  the  Gould  master  control  computer;  that  calculated  a  relative  position  and  velocity 
from  the  appropriate  missile  launch  station  to  the  target  aircraft. 

In  the  past,  the  HWIL  missile  used  pre-recorded  data  files  for  its  missile  umbilical  and  rear  data  link  (RDL) 
guidance  messages.  Now  in  order  to  process  a  near  real-time  ADS  stream,  the  MISILAB  had  to  acquire  a 
special  interface  unit  Hughes  Aircraft  Corp.  developed  it,  and  called  it  the  Advanced  Aircraft  Simulation 
Interface  (AASI).  The  AASI  controlled  the  AMRAAM  sequence  by  synchronizing  the  missile  data  to  the 
aircraft  TSPI  data  when  it  arrived  at  the  HWIL  input  ports,  then  injected  the  data  in  lockstep  with  the  missile 
simulation. 


1.6  Overview  of  Lessons  Learned 


All  of  this  background  information  sets  the  stage  for  the  topic  of  this  paper:  lessons  learned  executing  an 
ADS  test.  In  order  to  successfully  integrate  these  assets  for  a  near  real-time  test,  the  JADS  team  learned 
several  lessons  during  the  risk  reduction  and  test  execution  phases.  The  lessons  highlighted  here  concern  test 
control  aspects,  computer  processing,  and  telemetry  issues. 

Control  of  a  distributed  test  dealt  with  tactical  aircraft  control,  scenario  and  data  collection  decisions, 
collocation  of  critical  project  personnel,  voice  communications,  and  contingency  planning  for  real-time 
malfunctions.  Computer  processing  lessons  dealt  with  simulated  GPS  data,  pre-processing  live  GPS  data 
from  several  aircraft  pods,  and  creation  of  an  aircraft  to  HWIL-missile  interface.  Telemetry  issues  concerned 
aircraft  and  terrain  shielding,  and  an  implementation  to  handle  random  sensor  dropouts. 


2.  Lessons  Learned  in  Test  Control 

2.1  Tactical  Aircraft  Control 

Tactical  control  of  Eglin  aircraft  had  to  be  performed  at  the  Eglin  Central  Control  Facility,  which  proved  to  be 
the  optimal  location  for  other  key  project  controllers  and  directors.  This  was  significant  because  the  original 
concept  for  the  SIT  was  to  link  geographically  distributed  nodes  at  different  test  ranges.  In  the  original 
concept,  JADS  explored  the  possibility  of  controlling  the  test  from  a  remote  site,  e.g.,  either  one  of  the  ranges 
or  from  a  third,  neutral  site.  However,  upon  planning,  there  became  a  distinction  between  tactical  control  of 
live  aircraft,  and  scenario  control  of  test  events.  The  test  range  inexorably  maintained  tactical  control  of  their 
aircraft  over  their  airspace.  Further,  early  simulation  tests  found  a  2-3  second  delay  in  displaying  the  aircraft 
trajectories  at  a  remote  site;  these  early  findings  quashed  any  aircraft  control  alternatives  because  of  safety 
concerns.  The  real  lesson  was  that  the  range  insisted  on  control  of  their  aircraft,  so  the  ADS  implementation 
resulted  in  aircraft  control  from  the  range,  not  at  a  distributed  site. 

2.2  Project  and  Scenario  Control 

The  second  issue  was  where  to  locate  project  decision  makers  to  control  the  scenarios  and  test  events.  The 
lesson  was  that  getting  a  full  suite  of  monitors  and  communications  back  to  our  project  headquarters  was 
impractical,  so  project  control  was  performed  at  the  test  range.  During  early  risk  reduction  tests,  it  became 
apparent  that  when  handling  developmental  problems,  there  was  much  more  discussion  that  was  never 
transmitted  via  radio  links.  Much  troubleshooting  was  still  communicated  face-to-face,  or  over  local 
intercom,  or  via  direct  phone  line.  Even  with  some  display  monitors,  personnel  at  remote  sites  would  often 
hear  about  problems  late,  often  with  only  partial  explanations.  Therefore,  in  order  to  be  part  of  the  decision 
making  process,  the  test  director  had  to  be  collocated  with  the  instrumentation  controller  and  the  range 
aircraft  controller.  The  CCF  was  the  necessary  choice  to  be  centrally  located  with  the  critical  personnel,  all 
communication  networks,  and  with  the  majority  of  system  displays.  This  project  control  worked  adequately, 
but  could  still  be  improved  with  more  communication  (see  next  paragraph). 

2.3  Voice  Communication  Networks 

Upwards  of  30  distributed  people  necessitated  the  use  of  3  voice  communication  networks,  and  a  41'  network 
could  have  further  aided  decisionmaking.  See  Figure  2.3  for  a  communications  plan  diagram.  The  pilots  and 
aircraft  controllers  necessarily  used  a  UHF  radio  link,  and  no  other  personnel  transmitted  on  that  link. 
Another  network  linked  the  CCF  and  instrumentation  nodes  on  a  test  range  intercom  system.  In  order  to 
maintain  communication  discipline,  only  6  people  actively  transmitted  on  this  intercom,  while  all  other 
personnel  listened  on  speakers  and  interjected  when  necessary.  A  3d  network  linked  only  the  JADS  logger 


and  wide  area  network  personnel  via  teleconference  on  a  DSN  phone  line.  This  DSN  line  was  barely 
adequate  for  a  5-person  conference  call,  as  we  never  got  the  full  10  person  capability,  somebody  would 
randomly  get  cut  off  of  DSN  during  the  2-hour  mission,  and  not  all  people  could  hear  the  other  people 
adquately  (usually  one  person  had  to  repeat  run  calls,  and  we  often  bridged  two  conference  lines). 

Additionally,  in  post-mission  discussions,  project  personnel  agreed  that  a  4*  voice  network  just  for  the  4 
main  SIT  team  members  would  have  further  aided  decision  making.  Each  of  the  members  observed  some 
critical  part  of  the  network  puzzle,  and  in  a  wide  area  network,  no  one  person  had  all  of  the  information 
displayed  in  front  of  him.  Therefore,  a  4th  phone  or  intercom  network  would  have  helped  to  provide  more 
timely  inputs  between  project  personnel,  and  help  to  make  quicker  decisions  about  unforeseen  problems. 

2.4  Contingency  Planning 

Real-time  operation  required  more  contingency  planning  to  quickly  decide  on  alternatives.  A  great  benefit 
for  using  an  ADS  linked  network  was  having  several  "analysts  in-the-loop"  during  the  real-time  mission. 
The  test  director  had  to  make  timely  decisions  between  the  aircraft  passes  in  order  to  affect  the  outcome  or 
productivity  of  the  mission.  Initially,  a  small  list  of  Go  /  No  Go  criteria  was  made  before  the  first  live  flight. 
This  list  included  criteria  on  key  aircraft  components  (INS,  shooter  radar,  radios,  etc),  and  on  aircraft  pods  or 
ground  systems.  As  the  experience  level  increased  after  risk  reduction  missions,  this  list  was  expanded  to 
include  alternatives  in  case  of  failures  or  degraded  assets.  These  alternatives  included  changing  aircraft 
profiles,  timimg  of  maneuvers,  modes  of  the  GPS  pods,  computer  processor  swapouts,  etc.  Having  this 
contingency  plan  spelled  out  in  advance  truly  helped  the  project  director  make  rapid,  well-informed 
decisions  to  get  the  most  productive  use  out  of  the  remaining  mission  time. 


3.  Lessons  Learned  in  Computer  Processing 

3.1  Full  Linked  Ground  Simulation  not  Maturely  Developed  Yet 

The  linked  configuration  of  the  Eglin  PRIMES  facility  to  the  CCF  was  not  adequate  to  accomplish  our  ground 
rehearsal  objectives.  PRIMES  is  a  very  good  Installed  System  Test  Facility  (ISTF)  when  used  in  standalone 
mode,  but  there  were  some  software  interface  problems  in  the  ADS  linked  configuration.  Specifically,  the 
simulated  TSPI  inputs  were  not  adequate  to  properly  load  the  TSPI  Data  Processor  (TDP),  which  necessitated 
the  use  of  live  aircraft  for  future  integration  missions. 

The  three  TSPI  sources  were  to  be  received  and  pre-processed  in  the  CCF,  then  combined  in  the  TDP. 
However,  the  simulated  GPS  data  were  either  rejected  or  had  very  inaccurate  residual  errors.  The  simulated 
GPS  configuration  also  cannot  provide  differential  corrections,  nor  use  the  real-time  satellite  ephemeris  data; 
therefore,  only  the  pre-recorded  raw  GPS  measurements  from  almanac  data  were  used  for  a  GPS  solution. 
(Unresolved  interface  issues  narrow  down  to  the  exact  satellite  almanac  data  used  by  PRIMES,  the  origin 
used  by  the  GPS  pods,  and  a  probable  difference  in  satellite  modeling  used  by  PRIMES  and  the  TDP.)  The 
simulated  INS  data  was  somewhat  inaccurate  during  turns,  and  initially  had  problems  with  a  time  bias  and  a 
roll  problem. 

Although  the  vast  majority  of  linking  issues,  data  formats  and  aircraft  connections  are  fully  developed,  these 
few  TSPI  interface  problems  need  to  be  resolved  between  PRIMES  and  the  CCF.  Then,  this  will  provide  a 
valid  and  useful  ISTF  to  HWTL  simulation. 

3.2  Limited  Rate  for  GPS  Processing 

The  VAX  mainframe  computers  at  the  CCF  could  not  process  the  full  rate  of  GPS  data  from  4  pods,  so  the 
resulting  lesson  was  that  a  degraded  mode  of  operation  was  implemented  to  get  GPS  inputs.  As  explained 


earlier,  each  of  the  4  GPS  pods  transmitted  10  data  records  per  second  to  the  CCF,  and  the  CCF  also 
processed  ground  reference  measurements  in  order  to  make  differential  corrections.  The  JADS  team  was 
expecting  40  Hz  input  to  give  a  40  Hz  processed  output;  however,  in  practice,  only  12-15  Hz  was  achieved 
(about  3-4  Hz  from  each  pod).  The  limiting  factor  turned  out  to  be  a  computer  throughput  problem  of  the 
Multiple-object  Tracking  and  Control  System  (MTACS).  The  MTACS  is  a  network  of  VAX  mainframe 
computers  from  the  1980's  which  performs  GPS  processing  and  other  multi-lateration  functions. 

In  order  to  workaround  this  processing  limitation,  the  JADS  team  configured  the  GPS  pods  so  that  only  one 
pod  per  aircraft  would  transmit  its  10  Hz  data,  and  the  other  GPS  pod  onboard  was  in  standby  mode.  This 
configuration  allowed  the  MTACS  to  usually  provide  7-8  Hz  GPS  data  from  each  aircraft  This  rate  proved 
adequate  for  the  SIT  implementation  when  it  was  combined  with  other  TSPI  sensors  (discussed  later  in 
section  4.2). 

3.3  Special  Missile  Interface  Unit  Required 

A  special  purpose  interface  computer  was  necessary  to  link  the  aircraft  telemetry  to  the  HWIL  missile  in 
order  to  achieve  near  real-time  processing.  During  test  concept  meetings,  development  of  this  unique  asset 
was  not  planned  for,  so  the  lesson  was  that  JADS  had  to  slide  the  schedule  and  budget  for  this  interface 
computer.  This  interface  was  needed  to  provide  the  HWIL  missile  with  a  real-time  feed  of  initial  targeting 
information  and  midcourse  cueing  updates. 

As  stated  earlier,  Hughes  built  the  Advanced  Aircraft  Simulation  Interface  (AASI)  to  accept  near  real-time 
umbilical  and  rear  data  link  information,  and  synchronize  the  launch  event  to  the  aircraft  TSPI;  see  reference 
[4].  The  AASI  used  a  133  MHz,  Pentium-based  computer,  and  only  injected  a  necessary  subset  of  the 
AMRAAM  missile  parameters.  This  development  required  an  Interface  Control  Document  (ICD)  for  these 
parameters  between  the  CCF  and  the  MISILAB,  and  because  of  funding  and  scheduling  constraints,  the 
AASI  was  not  available  before  January,  1997.  Now,  a  few  AASI  units  are  available  and  verified  to  accept  the 
AMRAAM  telemetry  data  and  satisfy  the  near  real-time  run  requirement  for  ADS. 


4.  Lessons  Learned  in  Telemetry  Issues 

4.1  Telemetry  Dropouts  Required  Workarounds 

Early  testing  showed  that  even  short  INS  telemetry  dropouts  or  time  stamping  problems  caused  aborted  or 
invalid  passes;  workarounds  had  to  be  made  to  obtain  better  telemetry  reception.  The  lesson  was  that  the 
aircraft  configuration  and  range  airspace  usage  were  restricted  in  order  to  receive  consistent  telemetry. 

During  the  first  two  live  flight  risk  reduction  missions,  the  telemetry  or  time  problems  directly  caused  13 
invalid  passes  out  of  the  28  total  passes  attempted.  This  caused  a  few  workarounds  and  corrective  actions  to 
get  better  TM  reception. 

First,  the  two  wing  station  fuel  tanks  on  each  F-16  were  removed  in  favor  of  one  centerline  fuel  tank;  even 
this  tank  caused  unacceptable  shielding,  so  it  was  removed  in  favor  of  a  "clean  jet".  This  configuration 
provided  better  aircraft  telemetry  from  both  the  INS  and  the  simulated  missile  pod,  but  required  an  airborne 
refueling  after  about  45  minutes. 

Secondly,  analysis  revealed  that  east-west  profiles  (which  are  parallel  with  the  Eglin  coastline)  provided 
better  TM  reception  than  either  north-south  or  diagonal  profiles. 

Third,  the  aircraft  were  flown  at  higher  altitudes  than  originally  desired.  In  order  to  achieve  approximately 
line-of-sight  TM  reception,  the  aircraft  were  flown  above  18,000  feet  when  20-30  nautical  miles  offshore,  and 


above  21,000  feet  when  35-45  nautical  miles  offshore.  This  improved  air  to  ground  TM  reception,  and  also 
helped  the  FPS-16  radars  to  consistently  track  the  aircraft  (the  FPS-16s  reduce  multipath  effects  when  the 
aircraft  are  over  3.5°  above  their  horizon).  (Aside:  note  that  using  an  airborne  TM  relay  platform,  such  as  an 
E-9  aircraft,  was  out  of  scope  for  the  JADS  effort,  but  could  help  other  projects  which  have  low  altitude 
scenarios.) 

4.2  Merging  Several  TSPI  Sources  Advantageous 

Eglin  developed  a  TSPI  Data  Processor  (TDP)  to  merge  several  TSPI  inputs  in  near  real-time.  This  robust 
implementation  coasted  through  most  dropouts  to  allow  a  more  consistent  (and  ultimately  accurate)  solution 
of  entity  state  data. 

The  TDP  took  advantage  of  real-time  aircraft  telemetry  and  availability  of  real-time  GPS  pods  in  order  to 
calculate  more  accurate  kinematic  estimates.  In  the  past,  it  was  common  to  use  a  single  source  of  TSPI  on  the 
Eglin  range.  In  general,  the  FPS-16  ground  tracking  radars  can  be  very  inconsistent,  especially  in  altitude 
tracking,  and  thus  be  somewhat  inaccurate.  When  JADS  and  Dynetics  evaluated  the  original  TSPI  from  a  live 
shot,  the  FPS-16s  appeared  to  have  an  altitude  error  up  to  100's  of  meters.  Therefore,  a  more  consistent  and 
accurate  TSPI  solution  was  needed  to  drive  the  HWIL  simulation. 

The  resulting  lesson  was  that  the  TSPI  Data  Processor  could  provide  a  much  better  trajectory  estimate  by 
combining  several  sensor  inputs.  By  merging  the  aircraft  INS,  GPS,  and  FPS-16  data,  "the  TDP  solutions 
were  estimated  to  be  accurate  within  1-3  m  in  position  and  1  m/s  in  velocity",  (reference  [1])  Further,  even 
though  JADS  experienced  between  40-60%  dropouts  from  the  GPS  telemetry  data,  the  INS  and  radar  data 
were  available  over  99%  of  the  time.  Therefore,  "the  periodic  GPS  dropouts  did  not  significantly  degrade  the 
accuracy  of  the  position  solution,  because  the  TDP  used  the  accurate  INS  data  to  propagate  the  solution 
between  GPS  updates",  (reference  [1])  These  results  are  consistent  with  the  findings  of  Ball  Systems 
Engineering  Division  in  their  TDP  Final  Technical  Report  from  July  1994. 

Therefore,  by  improving  the  TSPI  accuracy  from  approximately  12  m  to  within  3  m,  it  allows  the 
measurement  tool  to  be  much  more  accurate  (by  an  order  of  magnitude)  than  a  typical  system  under  test 
The  impact  of  this  improved  accuracy  allows  a  missile  or  radar  analyst  to  make  unprecedented,  verified 
comparisons  of  missile  seeker  performance  or  aircraft  radar  tracking  performance. 
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SIT  Live  Fly  Data  Flow  Diagram 


SUMMARY  OF  COMM  NETS 
UHF  (279.70):  Sheriff  01,  Sheriff  02,  Chamber,  Project  (Wilkes); 
Rx-only:  Weekley. 
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Figure  2.3  SIT  Communications  Plan  Diagram 
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Building  an  Advanced  Distributed  Simulation  (ADS)  is  an  extremely  complex  process  for 
which  no  guidelines  currently  exist.  The  Modeling  and  Simulation  (M&S)  community  is 
rapidly  gaining  insight  into  the  ADS  structure  building  process  through  experience  in 
planning,  organizing,  and  conducting  several  large  ADS  exercises.  Experiences  have 
ranged  from  “very  good”  to  “very  bad”  with  no  one  area  of  the  ADS  building  process 
escaping  serious  pitfalls.  This  paper  attempts  to  summarize  the  experiences  encountered 
by  TACCSF  as  a  participant  in  several  ADS  projects.  Planning  is  the  key  to  success  and 
forms  the  basis  upon  which  a  successful  ADS  must  be  built.  Specific  areas  of  the  ADS 
process  covered  by  this  paper  are:  (1)  planning,  (2)  system  engineering,  (3)  connectivity, 
(4)  communications  security  (COMSEC),  (5)  scenario  development,  and  (6)  data 
collection  and  analysis.  The  “lessons  learned”  discussed  will  aid  the  ADS  planner  of  the 
future  avoid  the  well  traveled  pitfalls  discovered  by  past  travelers. 


Lessons  Learned 
in 

Building  ADS  Infrastructure 

INTRODUCTION 

Building  an  Advanced  Distributed  Simulation  (ADS)  infrastructure  is  an  extremely  complex  process.  There 
are  very  few  guidelines  and  most  territory  is  uncharted  water  for  any  organization  wanting  to  contribute  to 
joint  ADS  capabilities.  This  fact  must  be  emphasized;  no  one  organization  “builds”  the  ADS  infrastructure, 
rather  they  contribute.  Today,  we  want  to  share  insights  we  have  gained  through  experience  with  multiple 
ADS  ventures  (listed  below). 

-  Joint  Theater  Missile  Defense  Simulation  Network  (JTMDSN) 

-  Joint  Environment  for  Testing,  Training,  and  Analysis  (JETTA) 

-  Theater  Missile  Defense  Wargames  (TMD  WG) 

-  Joint  Air  Defense  Operations/Joint  Engagement  Zone  (JADO/JEZ) 

-  Advanced  Distributed  Simulation  Training  Exercises  (ADSTE) 

-  War  Breaker  (WB) 

-  Roving  Sands  95 

-  Air  Defense  Initiative  (ADI) 

Our  hope  is  that  where  we  have  approached  the  challenges  with  brute  force,  ingenuity,  and  even  ignorance, 
others  might  benefit  from  our  experience.  Together,  we  can  push  the  ADS  infrastructure  forward,  while 
avoiding  known  pitfalls  and  fostering  teamwork,  to  overcome  the  abundance  of  obstacles  awaiting 
discovery. 


PLANNING 

Planning  is  the  prerequisite  to  all  else.  It  determines  if  and  to  what  extent  you  succeed  in  building  and 
using  a  distributed  network.  Figure  1  illustrates  our  approach.  The  five  major  areas  of  System 
Engineering,  Connectivity,  COMSEC,  Scenario  Development,  and  Analysis,  capture  90%  of  the  ADS 
picture  which  must  be  understood  before  preceding.  These  five  areas  represent  concurrent,  interrelated 
activities,  not  sequential,  discrete  tasks.  When  orchestrated  toward  a  common  objective,  schedule,  and 
budget,  effective  enhancements  to  testing  and  training  capabilities  can  be  realized. 


PLANNING 


System  Engineering  Connectivity  COMSEC  Scenarios  Anaylsis 


TESTING  &  TRAINING 


Figure  1 


Planning  is  an  investment  in  success.  It  must  pervade  all  aspects  of  a  project,  address  coordination  of  all 
project  elements,  be  given  adequate  time,  and  must  continue  until  completion  of  a  project.  To  short-change 
planning  results  in  direct  cost  increases,  longer  timelines,  and  a  negative  work  environment  governed  by 
pressure,  frustration,  and  confusion  ...producing  less  than  optimum  results.  The  initial  investment  in  project 


planning  sets  the  direction  and  pace.  Continued  planning  maintains  the  direction  and  ensures  greater 
accuracy. 

Two  rules  govern  effective  planning.  The  first  is  the  one  most  projects  willingly  sacrifice  and  inevitably 
pay  for  it  later  -  planning  takes  time.  The  old  adage  -  “you  can  pay  me  now,  or  you  can  pay  me  later” 
applies  universally  to  ADS.  Our  experience  is  that  for  a  multi-organization  distributed  network,  one  year 
of  extensive  planning,  upfront,  is  an  excellent  rule  of  thumb  (ROT).  This  is  not  pragmatism,  it  is  reality. 
We’ve  violated  this  rule  numerous  times  to  meet  customer  timeline  demands.  Results  vary  from  pulling  out 
a  last  minute  miracle  to  delays  to  total  non-delivery.  But  when  there  is  dedicated  planning,  the  results  do 
not  vary  - 100%  success.  A  good  example  of  this  is  the  Combat  ID  project  with  Wright  Labs  (WL)  and 
Electronic  Support  Center  (ESC).  From  requirements  to  scenarios  to  technical  development  to  test  plan  to 
operational  relevancy,  we  formed  teams  that  planned  and  executed  systematically. 

The  second  ROT  which  relates  exponentially  to  the  number  of  distributed  participants  is  honesty.  All 
participants  must  be  honest  and  candid  about  why  they  are  participating  on  the  distributed  network,  and 
what  they  expect  to  gain  from  that  participation.  Experience  has  taught  us  -  most  customers  don’t  know 
exactly  what  ADS  can  or  can’t  provide,  and/or  aren’t  totally  honest  in  revealing  their  agenda.  That 
combination  clouds  and  misdirects  all  planning  efforts.  Additionally,  facilities  accurately  representing  their 
capabilities  and  estimates  of  feasible  development  are  essential  to  success.  Without  this  basic  trust, 
teamwork  is  impossible  ,and  thus  connectivity  of  distributed  simulations  will  be  superficial. 

The  following  discussion  presents  applications  of  the  planning  philosophy : 

-  Planning  is  bounded  and  guided  by  one  thing  -  OBJECTIVES.  What  are  they?  Is  this  a 
demonstration?  A  study?  What  are  the  goals?  What  are  the  Critical  Operational  Issues  (COIs)? 
Determining  objectives  is  a  sponsor/user  responsibility,  but  usually  requires  contribution  from  the  entire 
ADS  team. 

-  Establish  working  groups  (WGs)  for  the  major  functional  project  areas.  Functional  experts, 
empowered  to  make  decisions,  must  focus  their  group’s  taskings  while  ensuring  information  transfer 
between  groups.  As  a  minimum,  there  should  be  working  groups  to  cover  management  (POCs),  scenarios, 
test  planning  (data  collection/analysis),  systems  development/integration,  and  concept  of  operations 
(CONOPs)/rules  of  engagement  (ROE). 

-  For  WGs  to  function  as  cohesive  forces  there  must  be  structure.  To  borrow  from  Sun  Tzu  -  unity 
of  command  is  essential.  One  person  with  clear  authority  and  responsibility  must  lead  and  direct  each 
organization’s  single  POC  towards  the  commonly  agreed  upon  objectives.  We  refer  to  that  person  as  the 
project  hammer. 

-  Don’t  allow  development  to  begin  until  requirements  are  definitized  in  writing  and 
understood  by  those  who  have  to  meet  them.  Distributed  requirements  complicate  this  point,  but  do  not 
preclude  realizing  it .  It  cannot  be  emphasized  enough;  disciplined  leadership  and  responsibility  at  each 
organization  working  toward  fulfilling  what  has  been  agreed  upon  in  writing  is  a  basic  requirement.  This  is 
potentially  the  toughest  ADS  component  to  ensure  and  enforce. 

-  Establish  cut-off  dates  for  all  major  tasks.  Creating  an  integrated  schedule  is  not  sufficient;  it 
must  be  enforced.  Requirements  creep  or  excessive  software  “tweaking”  is  always  present  and  must  be 
dealt  with. 

-  Establish  a  configuration  control  management  mechanism  that  correlates  to  the  schedule.  The 
challenge  is  that  all  organizations  must  adhere  to  the  same  rules  100%  of  the  time  -  suggest  it  be  in  signed 
written  form. 

-  POCs  must  be  decision  capable;  workers  must  be  committed  to  tackling  the  hard  problems;  and 
growing  team  depth  in  areas  of  critical  expertise  must  be  inherent  to  the  plan.  Single  points  of  failure  are 
unacceptable  because  good  planning  can  prevent  it. 


In  summary,  plan  up-front,  plan  continually,  plan  for  success,  plan  for  failure,  and  then  plan  some  more. 


SYSTEM  ENGINEERING 

System  engineering  in  ADS  is  mainly  concerned  with  system  interfaces,  specifically  at  the  intersite  level. 

The  following  emphasizes  the  basics  underlying  solid  distributed  system  engineering. 

Time:  For  any  distributed  exercise,  it  always  takes  more  time  for  integration  than  is  expected.  Distributed 
exercises  have  an  inherently  high  number  of  variables,  and  they  always  have  some  that  you  can’t  predict.  A 
good  ROT  is  pad  your  scheduled  integration  time  by  50%.  Test  time  must  be  identified  to  occur  early  and 
often.  Connectivity  testing  must  take  a  building  block  approach,  i.e.,  adding  one  site  at  a  time.  It  is  the 
only  way  to  isolate  problems  and  control  the  variables. 

Communication:  Face-to-face  Technical  Interchange  Meetings  (TIMs)  are  far  more  beneficial  than  either 
Video  Teleconferences  (VTCs)  or  phone  calls.  The  key  is  use  any  and  all  communication  vehicles 
available,  but  show  preference  to  the  most  effective  means,  despite  costs.  The  key  is  to  coordinate  the 
system  engineering  activities  of  all  participants.  Use  of  the  video  conference  capabilities  on  development 
terminals,  such  as  the  Silicon  Graphics  Indy  system,  is  a  growing  communications  option. 

Requirements:  When  the  initial  requirements  for  a  distributed  exercise  are  laid  out,  they  are  usually  very 
loose  and  subject  to  interpretation.  This  is  in  direct  conflict  with  developer’s  need  at  each  site  to  truly 
understand  the  technical  tasks  before  tackling  them.  The  requirements  definition  process  is  customer 
initiated,  and  project  team  refined  -  it  should  be  a  technical  manifestation  of  the  operational  objectives.  The 
following  are  a  few  ROT  to  the  process: 

-  Describe  the  requirements  for  a  specific  node/system,  rather  than  to  simply  say  a  node  is  needed. 

It  is  impossible  for  the  project  “hammer”  to  know  the  level  of  detail  and  the  capabilities  of  each  simulation 
in  the  exercise;  therefore,  each  site  POC  should  be  responsible  for  providing  a  detailed  set  of  written 
requirements.  These  must  be  feasible  -  only  the  site  can  make  that  determination.  Non-delivery  of  a 
capability  has  a  domino  effect  on  ADS  exercises.  The  customer’s  objectives  and  budget  determine  the 
fidelity  of  the  solutions  (75%  solution?  100%  solution?). 

-  Once  each  site  has  honestly  defined  its  model  capability,  then  the  team  must  merge  these  into  a 
cohesive  technical  plan  which  satisfies  the  objectives.  The  result  should  be  clear  agreement  on  which 
simulations  are  to  be  used,  what  development  is  required,  and  what  the  criteria  are  for  adding  simulations 
during  the  development  phase.  Never  add  simulations  or  improvements  once  integration  testing  begins!! 

-  Experience  teaches  that  requirements  continue  to  grow  as  a  project  progresses.  Some  of  this 
growth  is  necessary  for  refinement  and  clarification,  but  often  it  is  fueled  by  a  site’s  hidden  agenda  or 
undisciplined  customer.  It  is  a  natural  desire  for  a  site  to  show  off  new  work  and  for  a  customer  to  get  all  he 
can  for  his  money,  but  uncontrolled  requirements  creep,  i.e.,  that  which  occurs  without  the  hammer’s 
approval,  will  result  in  constant  increases  and  schedule  delays.  Be  aware  -  fringe  elements  often  create 
havoc  in  an  otherwise  well-structured  exercise!  Distributed  discipline  is  gained  by  evaluating  the 
“perceived  new”  requirements  against  the  original  written  requirements,  and  fully  coordinating  any 
decisions  which  result  from  that  evaluation. 

DIS  implementation:  There  are  varying  degrees  and  approaches  to  DIS  implementation  which  equate  to 
integration  testing  land  mines .  It  is  critical  that  the  project  team  develop  a  formal  DIS  guidance  document 
for  two  reasons.  First,  to  establish  a  distributed  DIS  baseline,  i.e.,  2.0.3,  or  2.0.4,  and  prohibit  any  hybrid  or 
upgrade  in  progress  because  the  effects  are  unpredictable.  Second,  detail  which  PDUs  will  be  used  in  the 
exercise  and  cross-check  each  site’s  implementation  against  the  DIS  standard.  This  also  provides  a 
mechanism  to  work  issues  where  DIS  PDU  standards  don’t  exist. 


Another  problem  area  for  ADS  is  non-standard  database  usage;  for  example,  radar  cross  section  (RCS)  data. 
Since  DIS  does  not  transmit  RCS  data,  each  site  is  required  to  develop  its  own  parametric  data  set  for  the 
cross  section  of  an  entity.  The  values  of  this  type  of  data  are  almost  never  consistent  between  sites,  which 
equates  to  distributed  inconsistencies  -  whose  version  is  correct?  To  avoid  invalidating  test  results,  the  team 
should  review  the  requirements  to  determine  if  use  of  the  inconsistent  data  will  have  any  negative  impact.  If 
it  does,  then  the  team  must  agree  on  a  common  data  implementation;  if  not,  the  site  is  cleared  to  run  its  own 
version  with  everybody’s  blessing. 

Using  a  common  DIS  gateway  at  all  of  the  sites  on  the  network  is  one  way  we’ve  solved  many  ADS  system 
engineering  problems.  This  approach  was  taken  for  the  Joint  Theater  Missile  Defense  Simulation  Network 
(JTMDSN)  and  the  Joint  Evaluation  for  Training,  Testing,  and  Analysis  (JETTA)  programs.  Simplifying 
the  problem  this  way  is  not  always  cost  effective  because  it  requires  that  all  of  the  sites  integrate  their 
models  into  a  single  environment  generator  (EG). 


CONNECTIVITY 

Connectivity  refers  to  the  physical  Unking  of  individual  simulation  sites/faciUties  which  make  up  the  ADS 
architecture.  Common  means  of  linking  include  telephone  lines  (T-l  and  T-3  lines),  satellites,  Defense 
Simulation  Internet  (DSI),  and  common  STU-III  telephones.  Although  several  protocols  used  to  transmit 
information  are  in  use  today,  the  DIS  protocol  is  rapidly  being  estabhshed  as  the  standard.  The  following  is 
a  Ust  of  considerations  we  have  developed: 

-  Lead  times  to  establish  T-l  links,  encryption  procedures  (KG-194),  DSI  Network,  security 
Memorandum  of  Agreements  (MOAs),  and  supporting  Hnk  requirements  usually  require  a  minimum  of  90 
days,  but  may  require  as  much  as  six  months.  Hardware  requirements  (interfaces,  routers,  etc.)  will  require 
at  least  four  months  from  date  of  request  to  actual  hardware  in  hand.  Security  procedures/MOAs  are 
required  by  government  directive.  Suffice  to  say,  these  are  total  show-stoppers  that  must  be  addressed 
during  initial  ADS  planning. 

-  System  Loading:  Each  type  of  physical  link  is  capable  of  transmitting  data  at  a  specific  rate.  As 
the  rate  increases,  so  does  cost.  Cost  savings  can  be  quite  substantial  if  the  time  is  taken  up  front  to  match 
load  requirements  with  links  available  (T-l,  T-3,  STU-III,  normal  phone  line).  Figure  2  depicts  a  low-cost 
configuration  used  during  Roving  Sands  to  link  low-bandwidth  users.  Additional  consideration  is  that  some 
KG  devices  serve  as  system  choke-points.  Proven  bench  marks  indicate  BReeze  boxes  can  handle  50 
dynamic  entities,  DSI  (NESS)  -  currently  100,  SATCOM  -  400,  and  T-ls  -  2000. 


-  Equipment:  Purchase  and  use  prebuilt,  tested  equipment  (including  connecting  cables)  whenever 
possible.  Untested,  built-on-the-fly  parts  will  definitely  cause  major  problems. 

-  DIS  limitations:  DIS  Protocol  Data  Units  (PDUs)  are  multicast  in  packets  and  the  technology  to 
route  these  packets  does  not  exist.  Bridging  is  the  only  capability  currently  available  to  connect  remote 
sites,  i.e.,  any  site  putting  PDUs  on  the  network’s  DIS  LAN  is  effectively  broadcasting  to  the  world. 


Ineffective  DIS  PDUs  can  quickly  flood  the  network.  Additionally,  multi-project  users  of  a  single  DIS 
network  often  unknowing  impact  each  other.  Already,  there  is  a  need  for  coordinated  scheduling  of  ADS 
networks. 


COMSEC 

Communications  Security  (COMSEC)  is  not  widely  understood  or  appreciated  by  the  majority  of  persons 
involved  in  developing  an  ADS  infrastructure.  Mixing  of  classified  and  unclassified  information  must  be 
avoided.  If  a  site  on  the  network  is  transmitting  classified  data  relating  to  a  model,  a  scenario,  CONOPs, 
ROEs,  etc.,  all  participants  must  be  practicing  COMSEC  procedures  compatible  with  the  highest  Security 
classification  of  the  material  involved.  The  consequences  of  ignoring  COMSEC  are  very  serious  -  a  breach 
of  National  Security  may  certainly  occur;  or  closer  to  home,  your  organization  may  be  shut  down.  The 
following  are  the  COMSEC  issues  requiring  early  and  vigilant  attention: 

-  An  MOA  defining  COMSEC  procedures  to  be  practiced  by  all  participants  during  an  ADS 
project  is  a  government  requirement.  An  MOA  between  new  sites  typically  requires  at  least  90  days  to 
negotiate,  sign,  and  be  approved. 

-  The  equipment  and  supporting  material  (such  as  keys  and  establishing  accounts)  required  to 
implement  COMSEC  will  require  a  minimum  of  60  days  from  request  to  actual  material  delivery. 

-  Multi-level  security  filters  require  National  Security  Agency  (NSA)  approval.  This  entails  a  three 
to  six  month  process,  acquiring  NSA  approved  system  and  certification. 


SCENARIO  DEVELOPMENT 

The  scenario  is  a  key  element  vital  to  the  success  of  an  ADS  project.  The  scenario  working  group  must 
continuously  communicate  with  all  other  participants  (working  groups)  to  insure  the  scenario  is  consistent 
with  the  parameters  defining  the  ADS  exercise.  The  following  list  identifies  several  areas  which  must  be 
addressed  during  scenario  development. 

-  The  Scenario  Group  must  remain  cognizant  of  test  objectives  and  COIs  and  build  the  scenario  to 
provide  necessary  situations  and  events.  A  scenario  that  does  not  address  the  objective  and  COIs  is  doomed 
for  failure. 

-  Scenario  specific  requirements  (theater  of  operations,  timeframe,  platforms,  tactics,  etc.)  must  be 
well-defined  upfront  with  fewer  and  fewer  changes  as  the  project  progresses.  These  needs  should  take  into 
consideration  the  strengths  and  limitations  of  the  total  architecture  and  the  risks  associated  with  new 
development. 

-  The  Scenario  Group  should  be  established  with  representatives  from  sites,  to  include  the 
operational  customer.  This  group  will  establish  and  define  scenario  elements  required  to  meet  test 
objectives  and  requirements.  Members  should  also  be  very  familiar  with  the  unique  capabilities  at  their 
own  site,  and  how  these  will  play  in  the  system  architecture.  Division  of  tasks  should  allow  each  site  to 
contribute  to  the  overall  scenario.  Connectivity  and  communications  between  exercise  participants  and 
models  should  be  set  up  early  and  exercised  regularly. 

-  As  accurately  as  possible,  the  scenario  should  present  a  realistic  set  of  conditions  to  the  test 
participants;  the  scenario  should  strike  a  balance  between  triviality  and  robustness.  In  addition  to 
developing  the  exercise’s  operational  flavor,  the  Scenario  Group  should,  during  the  course  of  scenario 
reviews,  be  cautious  to  remove  as  many  non-real  world  situations  as  possible.  The  group  should  be 
cognizant  of  “sim-isms,”  which  could  affect  test  results  or  user  credibility  in  the  scenario.  A  Crew  Aid 
discussing  CONOPS,  ROE,  specific  airspace,  special  instructions,  and  a  connectivity  diagram  is  key  for  all 
operators.  Subsets  of  the  actual  exercise  should  be  tested  with  crews  similar  to  those  who  will  participate  in 


the  final  test  to  help  identify  unrealistic  conditions  or  other  items  the  test  designers  may  have  overlooked. 
These  pre-trial  runs  should  exercise  as  much  of  the  system  architecture  as  is  available,  to  include  data 
collection  and  processing  for  the  final  report. 

-  All  sites  contributing  to  the  scenario  must  agree  on  a  date  when  the  scenario  will  be  frozen.  This 
date  should  be  established  far  enough  in  advance  of  the  experiment  to  allow  for  adequate  testing,  but  not  so 
far  ahead  that  the  system  is  still  immature.  Changes,  if  allowed,  should  be  thoroughly  tested  for  adverse 
impacts  on  the  full  system.  Strict  configuration  management  should  be  enforced  to  adequately  understand 
the  impacts  at  each  site;  regression  testing  should  be  performed  in  a  logical  order  to  isolate  which  site  has 
corrupted  the  network. 

-  “Reality  checks”  should  be  conducted  frequently  to  ensure  realism,  adequate  addressing  of  test 
objectives,  and  the  elimination  of  “sim-isms.”  The  timeliness  of  these  evaluations  cannot  be 
overemphasized:  as  soon  as  the  system  matures,  testing  should  begin.  Testing  should  exercise  all 
subsystems  to  be  used  during  the  actual  test;  “train  like  you  plan  to  test.” 


DATA  COLLECTION  AND  ANALYSIS 

DIS  PDUs  provide  a  common  source  of  data  collection  for  all  ADS  network  sites,  which  complements  any 
local  data  collection  performed  at  the  various  models  and  simulators.  Logging  of  PDUs  with  such  tools  as 
the  STRICOM  Logger  is  most  useful  for  system  debug  and  integration  testing.  The  TACCSF  Trial  Event 
Generator  (TEG),  in  addition,  provides  selection  of  PDUs  with  analytic  interest,  on-line  merging  of 
operator  and  system  perceived  events  with  entity  truth  data,  and  efficient  storage  of  structured  data  records 
for  analysis.  These  TEG-produced  Trial  Events  can  be  processed  by  a  commercial  analytic  software 
package  called  PV  WAVE  or  can  be  loaded  directly  into  a  spreadsheet  to  compute  test  measures  or  plot 
timelines  of  critical  operational  events  and  network  performance  measures.  Analytic  software  is  being 
developed  which  can  process  these  Trial  Events  and  display  mission  performance  measures  in  near  realtime. 
The  most  challenging  aspect  of  ADS  data  collection  and  analysis  is  the  generation  of  data  collection  PDUs 
by  participating  models  and  simulators  in  order  to  capture  key  tactical  events.  Key  entity  truth  events  are 
already  being  captured  by  means  of  Entity  State,  Fire,  and  Detonate  PDUs.  A  timeline  analysis  for  Theater 
Ballistic  Missile  (TBM)  attack  operations  could  be  produced  from  the  following  data  collection  sequence, 
for  example: 


EVENT 

DATA  SOURCE 

PDU 

Create  TEL 

TEL 

Entity  State 

TBM  launch 

TEL 

Fire 

TBM  detect 

DSP,  TPS-75,  or  Cobra 

Ball  Event  Report 

Launch  Point  Est.  DSP, 

Correlator,  or  Cobra  Ball 

Event  Report 

Joint  STARS  tasked 

CRC  or  AOC 

Manual 

Allocate  AW  ACS 

CRC  or  AOC 

Manual 

Commit  F-15E 

AWACS 

Manual 

TEL  moves 

TEL 

Entity  State 

Joint  STARS  update 

Joint  STARS 

Manual 

F-15E  detect  TEL 

F-15E 

Manual 

EVENT 

DATA  SOURCE 

PDU 

F-15E  engage  TEL 

F-15E 

Fire 

Weapon  impact 

F-15E 

Detonate 

TEL  destroyed 

TEL 

Entity  Stage 

In  addition  to  expanded  data  generation  from  network  participants,  other  requirements  for  effective  DIS 
PDU  data  collection  and  analysis  include: 


-  Universal  implementation  of  START/STOP  PDUs  to  assist  in  exercise  control  and 
synchronization  across  sites  using  standard  Global  Positioning  System  (GPS)  clock  time. 

-  Improved  enumeration  management. 

-  Replace  manual  data  collection  with  digital  data  collection  wherever  possible. 

-  Unified  operational  debriefs  for  geographically  separated  sites. 

-Time  stamps  for  Entity  State  PDUs  must  allow  for  values  greater  than  one  hour. 


CONCLUSION 

The  number  of  ADS  builders  and  users  is  growing  rapidly.  The  age  of  ADS  is  here  regardless  if  people  or 
the  technology  are  ready  for  it.  Therefore,  the  learning  curve  is  steep  and,  as  constructive,  virtual,  and  live 
environments  merge,  the  grade  doesn’t  show  any  sign  of  leveling  out.  The  purpose  of  this  paper  is  to 
communicate  that  the  ADS  challenges  are  global  and  unless  there  is  an  open  sharing  of  the  good,  the  bad, 
and  the  ugly,  the  learning  curve  will  remain  painfully  steep  for  all.  There  must  be  a  distributed  mindset 
committed  to  planning.  The  project  leadership  principles,  considerations,  and  common  sense  reminders 
provide  a  starting  point  to  building  a  functional  ADS  infrastructure.  The  challenge  is  to  tackle  the  issues  of 
system  engineering,  connectivity,  COMSEC,  scenario  development,  data  collection,  and  analysis  as  a 
distributed  team  focused  on  the  project’s  objectives  and  equipped  with  a  winning  plan. 
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